Diseño e implementación de un sistema de ficheros distribuido basado en Memcached

En la actualidad, 18 de los 20 sitios web más importantes en lo que a número de usuarios se refiere, utilizan Memcached para mejorar su rendimiento y escalabilidad (por ejemplo, Google, YouTube, Facebook, Twitter, etc.). En este Proyecto de Fin de Carrera se hará una aproximación para estudiar la viabilidad de utilizar Memcached como base para el diseño e implementación de un sistema de ficheros distribuido de alto rendimiento para entornos clúster. En este proyecto se pretende obtener una interfaz para trabajar con Memcached como si de un sistema de ficheros distribuido se tratase, permitiendo incluso, utilizar Memcached como caché para un sistema de ficheros local, siempre procurando que la interfaz diseñada e implementada sea lo más similar posible a la ofrecida por POSIX para las operaciones de entrada y salida estándar (open, read, write, close, etc.) Además en este proyecto se presenta la evaluación de rendimiento de todas las funcionalidades implementadas, de modo que se pueda juzgar la viabilidad y utilidad de un sistema de ficheros distribuido basado en Memcached. _____________________________________________________________________________________________________________Nowadays, 18 of the top 20 sites in terms of user counts are powered by Memcached to improve their performance and scalability (e. g. Google, YouTube, Facebook, Twitter, etc.). In this end-of-degree dissertation an approximation to the study of the viability of using Memcached as the base of the design and implementation of a distributed file system will be done. This work aims to obtain an interface (an API) for working with Memcached as a distributed file system and utilizing Memcached as a local file system‟s cache, always trying that the API designed and implemented is as similar as possible to the one used in POSIX for standard I/O operations (open, read, write, close, etc.) We have evaluated the performance of each one of the implemented functionalities and with the data obtained, evaluated the viability and utility of a distributed file system based in Memcached.Ingeniería en Informátic

A generic I/O architecture for data-intensiveapplications based on in-memorydistributed cache

Proceedings of the First PhD Symposium on Sustainable Ultrascale Computing Systems (NESUS PhD 2016) Timisoara, Romania. February 8-11, 2016.The evolution in scientific computing towards data-intensive applications and the increase of heterogeneity in the computing resources, are exposing new challenges in the I/O layer requirements. We propose a generic I/O architecture for data-intensive applications based on in-memory distributed caching. This solution leverages the evolution of network capacities and the price drop in memory to improve I/O performance for I/O-bounded applications adaptable to existing high-performance scenarios. We have showed the potential improvementsEuropean Cooperation in Science and Technology. COSTThis work is partially supported by EU under the COST Program Action IC1305: Network for Sustainable Ultrascale Computing (NESUS). This work is partially supported by the grant TIN2013-41350-P, Scalable Data Management Techniques for High-End Computing Systems from the Spanish Ministry of Economy and Competitiveness

CoSMiC: A hierarchical cloudlet-based storage architecture for mobile clouds

Storage capacity is a constraint for current mobile devices. Mobile Cloud Computing (MCC) is developed to augment device capabilities, facilitating to mobile users store/access of a large dataset on the cloud through wireless networks. However, given the limitations of network bandwidth, latencies, and devices battery life, new solutions are needed to extend the usage of mobile devices. This paper presents a novel design and implementation of a hierarchical cloud storage system for mobile devices based on multiple I/O caching layers. The solution relies on Memcached as a cache system, preserving its powerful capacities such as performance, scalability, and quick and portable deployment. The solution targets to reduce the I/O latency of current mobile cloud solutions. The proposed solution consists of a user-level library and extended Memcached back-ends. The solution aims to be hierarchical by deploying Memcached-based I/O cache servers across all the I/O infrastructure datapath. Our experimental results demonstrate that CoSMiC can significantly reduce the round-trip latency in presence of low cache hit ratios compared with a 3G connection even when using a multi-level cache hierarchy

Different aspects of workflow scheduling in large-scale distributed systems

As large-scale distributed systems gain momentum, the scheduling of workflow applications with multiple requirements in such computing platforms has become a crucial area of research. In this paper, we investigate the workflow scheduling problem in large-scale distributed systems, from the Quality of Service (QoS) and data locality perspectives. We present a scheduling approach, considering two models of synchronization for the tasks in a workflow application: (a) communication through the network and (b) communication through temporary files. Specifically, we investigate via simulation the performance of a heterogeneous distributed system, where multiple soft real-time workflow applications arrive dynamically. The applications are scheduled under various tardiness bounds, taking into account the communication cost in the first case study and the I/O cost and data locality in the second.The work presented in this paper has been partially supported by EU, under the COST program Action IC1305, “Network for Sustainable Ultrascale Computing (NESUS)”, and by the Ministerio de Economía y Competitividad, Spain, under the project TIN2013-41350-P, “Scalable Data Management Techniques for High-End Computing Systems”

Evaluating data caching techniques in DMCF workflows using Hercules

The Data Mining Cloud Framework (DMCF) is an environment for designing and executing data analysis workflows in cloud platforms. Currently, DMCF relies on the default storage of the public cloud provider for any I/O related operation. This implies that the I/O performance of DMCF is limited by the performance of the default storage. In this work we propose the usage of the Hercules system within DMCF as an ad-hoc storage system for temporary data produced inside workflow-based applications. Hercules is a distributed in-memory storage system highly scalable and easy to deploy. The proposed solution takes advantage of the scalability capabilities of Hercules to avoid the bandwidth limits of the default storage. Early experimental results are presented in this paper, they show promising performance, particularly for write operations, compared to the performance obtained using the default storage services.This work is partially supported by EU under the COST Program Action IC1305: Network for Sustainable Ultrascale Computing (NESUS). This work is partially supported by the grant TIN2013-41350-P, Scalable Data Management Techniques for High-End Computing Systems from the Spanish Ministry of Economy and Competitiveness

Exploiting data locality in Swift/T workflows using Hercules

The ever-increasing power of supercomputer systems is both driving and enabling the emergence of new problem-solving methods that require the efficient execution of many concurrent and interacting tasks. Swift/T, as a description language and runtime, offers the dynamic creation and execution of workflows, varying in granularity, on high-component-count platforms. Swift/T takes advantage of the Asynchronous Dynamic Load Balancing (ADLB) library to dynamically distribute the tasks among the nodes. These tasks may share data using a parallel file system, an approach that could degrade performance as a result of interference with other applications and poor exploitation of data locality. The objective of this work is to expose and exploit data locality in Swift/T through Hercules, a distributed in-memory store based on Memcached, and to explore tradeoffs between data locality and load balance in distributed workflow executions. In this paper we present our approach to enable locality-based optimizations in Swift/T by guiding ADLB to schedule computation jobs in the nodes containing the required data. We also analyze the interaction between locality and load balance: our initial measurements based on various raw file access patterns show promising results. Moreover, we present future work based on the promising results achieved so far.This material is based upon work supported by the U.S. Department of Energy, Office of Science, under contract DE-AC02-06CH11357. Computing resources were provided by the Argonne Leadership Computing Facility. The work presented in this paper was supported by the COST Action IC1305, “Network for Sustainable Ultrascale Computing (NESUS).” The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement number 328582

A Data-Aware Scheduling Strategy for DMCF workflows over Hercules

Proceedings of: Third International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2016). Sofia (Bulgaria), October, 6-7, 2016.As data-intensive scientific prevalence arises, there is a necessity of simplifying the development, deployment, and execution of complex data analysis applications. The Data Mining Cloud Framework is a service-oriented system for allowing users to design and execute data analysis applications, defined as workflows, on cloud platforms, relying on cloud-provided storage services for I/O operations. Hercules is an in-memory I/O solution that can be deployed as an alternative to cloud storage services, providing additional performance and flexibility features. This work extends the DMCF-Hercules cooperation by applying novel data placement and task scheduling techniques for exposing and exploiting data locality in data-intensive workflows.This work is partially supported by EU under the COST Program Action IC1305: Network for Sustainable Ultrascale Computing (NESUS)

La evaluación en el contexto del Espacio Europeo de Educación Superior.

La evaluación es un elemento que preocupa siempre a todos los implicados en el proceso educativo (profesores, alumnos, responsables académicos, etc.). Sin embargo, la evaluación es un concepto muy amplio y no todas sus facetas preocupan de igual manera ni reciben la misma atención. Evaluar, desde luego, es asignar una calificación que refleje el nivel de aprendizaje. Pero evaluar es también identificar lo que debe mejorarse de cara al curso siguiente. En nuestro artículo, partiendo del modelo de Donald Kirkpatrick sobre los diferentes enfoques de la evaluación, describimos las estrategias y técnicas que utilizamos en la actualidad para desplegar un sistema de evaluación de amplio alcance, en el contexto del proceso de adaptación de nuestras asignaturas de primeros cursos de ingeniería a los requerimientos del Espacio Europeo de Educación Superior (EEES). Precisamente en el contexto del EEES algunos enfoques de la evaluación a los que normalmente se presta poca atención, como es el caso de la evaluación del seguimiento de la actividad del alumno, adquieren una relevancia muy importante.Postprint (published version

El Puzle i l'aprenentatge de la programació

Es descriu una experiència formalitzada d’aprenentatge cooperatiu mitjançant la utilització de la metodologia del puzle. Concretament, s’utilitza el puzle com una eina de suport en una assignatura organitzada d’acord amb l’estratègia de l’aprenentatge basat en projectes. La metodologia del puzle s’ha aplicat a tots els grups de les assignatures següents des del curs 2004-2005: 1. Enginyeria Tècnica de Telecomunicació, Introducció als Computadors (Q1 de primer curs, 4,5 crèdits) i Laboratori de Programació (Q2 de primer curs, 4,5 crèdits), i 2. Graus en Enginyeria de Sistemes de Telecomunicació i en Enginyeria Telemàtica, Introducció als Ordinadors (Q1 de primer curs, 6 crèdits ECTS) i Projecte de Programació (Q2 de primer curs, 6 crèdits ECTS). Com a part d’assignatures basades en l’aprenentatge basat en projectes es va començar a experimentar en dues assignatures a l’Escola d'Enginyeria de Telecomunicació i Aeroespacial de Castelldefels (EETAC), en el marc de les proves pilot d’adaptació de titulacions a l'espai europeu d'educació superior (EEES), impulsades per l’antic Departament d'Universitats, Recerca i Societat de la Informació (DURSI) de la Generalitat de Catalunya. Posteriorment es va seguir aplicant aquest model en els nous graus. Aquests anys d’experimentació ens han proporcionat una bona quantitat de resultats i conclusions.Peer Reviewe

Reducing the environmental impact of surgery on a global scale: systematic review and co-prioritization with healthcare workers in 132 countries

Abstract Background Healthcare cannot achieve net-zero carbon without addressing operating theatres. The aim of this study was to prioritize feasible interventions to reduce the environmental impact of operating theatres. Methods This study adopted a four-phase Delphi consensus co-prioritization methodology. In phase 1, a systematic review of published interventions and global consultation of perioperative healthcare professionals were used to longlist interventions. In phase 2, iterative thematic analysis consolidated comparable interventions into a shortlist. In phase 3, the shortlist was co-prioritized based on patient and clinician views on acceptability, feasibility, and safety. In phase 4, ranked lists of interventions were presented by their relevance to high-income countries and low–middle-income countries. Results In phase 1, 43 interventions were identified, which had low uptake in practice according to 3042 professionals globally. In phase 2, a shortlist of 15 intervention domains was generated. In phase 3, interventions were deemed acceptable for more than 90 per cent of patients except for reducing general anaesthesia (84 per cent) and re-sterilization of ‘single-use’ consumables (86 per cent). In phase 4, the top three shortlisted interventions for high-income countries were: introducing recycling; reducing use of anaesthetic gases; and appropriate clinical waste processing. In phase 4, the top three shortlisted interventions for low–middle-income countries were: introducing reusable surgical devices; reducing use of consumables; and reducing the use of general anaesthesia. Conclusion This is a step toward environmentally sustainable operating environments with actionable interventions applicable to both high– and low–middle–income countries